Bag of Optical Flow Volumes for Image Sequence Recognition1

This paper introduces a novel 3D interest point detector and feature representation for describing image sequences. The approach considers image sequences as spatiotemporal volumes and detects Maximally Stable Volumes (MSVs) in efficiently calculated optical flow fields. This provides a set of binary optical flow volumes highlighting the dominant motions in the sequences. 3D interest points are sampled on the surface of the volumes which balance well between density and informativeness. The binary optical flow volumes are used as feature representation in a 3D shape context descriptor. A standard bag-of-words approach then allows building discriminant optical flow volume signatures for predicting class labels of previously unseen image sequences by machine learning algorithms. We evaluate the proposed method for the task of action recognition on the well-known Weizmann dataset, and show that we outperform recently proposed state-of-the-art 3D interest point detection and description methods.